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Abstract 


In-person college advising programs generate large improvements in college persistence 
and success for low-income students but face numerous barriers to scale. Remote advising models 
offer a promising strategy to address informational and assistance barriers facing the substantial 
majority of low-income students who do not have access to community-based advising, yet the 
existing evidence base on the efficacy of remote advising is limited. We present a comprehensive, 
multi-cohort experimental evaluation of CollegePoint, a national remote college advising program 
for high-achieving low- and moderate-income students. Students assigned to CollegePoint are 
modestly more likely (1.3 percentage points) to attend higher-quality institutions. Results from 
mechanism experiments we conducted within CollegePoint indicate that moderate changes to the 
program model, such as a longer duration of advising and modest expansions of the pool of 
students academically eligible to participate, do not lead to larger program effects. We also 
capitalize on across-cohort variation in whether students were affected by COVID-19 to 
investigate whether social distancing required by the pandemic increased the value of remote 
advising. CollegePoint increased attendance at higher-quality institutions by 3.2 percentage points 
for the COVID-19-affected cohort. 
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L Introduction 


Federal and state education policy increasingly prioritizes college access for low-income student 
populations. This is most evident in the growing number of “free college” programs at the state 
level (e.g. Tennessee Promise) and the proposed expansion of free college nationally through the 
Biden Administration American Families Plan. Yet evidence suggests that financial barriers are 
not the sole driver of inequalities in completion by family income (Bulman et al., 2017). 
Informational barriers, complex college and financial aid application processes, and a lack of 
access to professional counseling can impede students from attending higher-quality institutions 
where they are more likely to earn a degree and from receiving financial aid for which they would 
qualify (Bowen, Chingos, and McPherson, 2009; Dynaski and Scott-Clayton, 2004; Hoxby and 


Avery, 2012). 


Traditionally the strongest lever that communities and high schools have for overcoming these 
barriers is to provide students with in-person college and financial advising. In-person advising 
programs generally either place an advisor within a school or have an office in the community for 
students to visit and work with an advisor. Several experimental evaluations of in-person college 
advising and mentoring programs find large and, in some cases, sustained impacts on low-income 
students’ college outcomes (Bettinger and Evans, 2019; Bos et al., 2012; Carrell and Sacerdote, 
2017; Castleman, Page, and Schooley, 2014). Two recent experimental evaluations of intensive 
advising programs, which provide individualized advising starting in high school and continuing 
in college, find large impacts on persistence several years into college (Castleman, Deutschlander, 


and Lohner, 2020; and on degree attainment (Barr and Castleman, in progress). 


While in-person advising programs can be successful they are difficult to scale and often do not 
serve areas with lower population density. In-person models rely on local relationships and 
individual agreements with high schools. Expanding to a new community requires relationship 
building, staffing, and other sources of infrastructure that may be difficult to develop in other 
communities. Organizations that operate outside of high schools face additional limitations to 
expansion. Since students travel to these advising organization, they locate near the students they 
plan to serve and in areas with many potential students. Because of the student density 
requirements, community-based advising organizations tend to locate in urban areas and have 
limited student draw areas.' As a result, many regions of the country are too dispersed to support 


investment in a community-based college advising organization. 


In the face of these scale challenges, there has been growing interest among policymakers and 
educators in the potential for remote advising models to provide a scalable strategy for supporting 
low-income and first-generation college students through the complexities of the college and 
financial aid application processes. Generally speaking, remote advising models use interactive 
technologies (phone, text messaging, video conference, screen sharing, and document 
collaboration) to facilitate sustained advising relationships between students and advisors even if 
they live far apart from each other. The main advantage of the remote model over the community- 
based model is the ability to reach students across the country with a limited geographic footprint. 


Therefore, the remote advising model could serve students from regions that are not dense enough 


' As an indication of the scale challenges, even community-based advising programs with experimental evidence of 
large impacts on college enrollment and success have only scaled modestly. For instance, Bottom Line has operated 
for more than twenty years in Boston and Worcester, MA. Even with experimental evidence of large impacts on 
student outcomes, the program has only expanded to date to New York and Chicago. 


to support a community-based organization while also increasing access to advising in densely 


populated areas because students no longer need to commute to receive advising. 


In this paper, we present a comprehensive experimental evaluation of a nationwide, virtual college 
advising program. CollegePoint is a Bloomberg Philanthropies-funded national remote advising 
program that since 2014 has served tens of thousands of students. A consortium of four advising 
organizations provides advising to the high-achieving, low- and moderate-income high school 
seniors participating in CollegePoint.” The primary objective of CollegePoint is to increase the 
share of this population who enrolls at high-quality colleges and universities, which CollegePoint 
defines as institutions with graduation rates above 70 percent. Throughout the remainder of the 


paper, we refer to these institutions as “CollegePoint schools.” 


We conducted a multi-cohort randomized controlled trial (RCT) of CollegePoint. We randomly 
assigned eligible students who applied to participate in CollegePoint to either a treatment group 
who received advising or a control group that did not receive CollegePoint advising.*? Our 
experimental sample consists of approximately 25,700 students from the high school graduating 
classes of 2018 - 2020. To preview our results, we find that students assigned to CollegePoint 
advising were 1.3 percentage points (2.5 percent) more likely to enroll at CollegePoint schools. 
This impact is primarily driven by increasing the share of students who apply to and attend 


Barron’s | colleges and universities--the most selective institutions in the country.* The effect on 


2 CollegePoint defines high-achieving, low- and moderate income as students in the top 10 percent of the SAT 
distribution nationally, with family income under $80,000 

3 Ina parallel qualitative study commissioned by Bloomberg Philanthropies, control group students reported little 
access to formal college advising and were largely independent in the college exploration and planning process. 

4 Like Smith (2018), we proxy for college applications using SAT score sends. Smith (2018), Card and Krueger 
(2005) and Pallais (2015) argue score sends are a good proxy for student application patterns. 


initial enrollment at Barron’s | institutions persists into students’ second year of college (for the 
students from cohorts 1 and 2) and into students’ third year (for the students from cohort 1). That 
being said, treated students are no more likely to persist at CollegePoint schools overall or to 


remain continuously enrolled in college (either overall or at CollegePoint schools specifically). 


Using additional sources of within-treatment experimental variation, we explore the impact of 
three potential mechanisms of CollegePoint’s efficacy. First, capitalizing on random assignment 
to advising organizations for a subset of students, we observe larger impacts among the students 
assigned to an organization that used peer mentors with small caseloads than students assigned to 
an organization that used full-time recent college graduates with much larger caseloads. Second, 
we do not find evidence that a longer duration of engagement with CollegePoint advising increases 
the program’s efficacy. Students randomly assigned to begin CollegePoint advising in the spring 
of their junior year were no more likely to enroll at a CollegePoint school than students assigned 
to begin CollegePoint advising at the start of their senior year in high school. Third, remote 
advising did not increase attendance at CollegePoint schools among a pilot cohort of students 


recruited from modestly lower in the academic distribution (the 85th - 89th percentile). 


Our paper builds on to two other recent experimental evaluations of remote college advising 
programs. Our analysis of CollegePoint is similar to Phillips and Reber (2019)’s analysis of the V- 
SOURCE program in Los Angeles in our multi-cohort design, ability to measure impacts on 
persistence, and our investigation of mechanisms. We build on Phillips and Reber (2019) by 
investigating a national remote advising program and enhancing the investigation of key program 


components through within-CollegePoint experiments. Our paper is similar to Gurantz et al. 


(2019) in our investigation of a national advising program but builds on this paper through our 


multi-cohort design, investigation of persistence impacts, and mechanisms experiments. 


Our paper makes an additional unique contribution beyond prior studies of remote advising 
interventions: One of our experimental cohorts overlaps with the COVID-19 crisis. More 
specifically, students in the third experimental cohort were in March of their senior year of high 
school--and still part of the CollegePoint advising program--when the COVID-19 pandemic began. 
This enables us to compare CollegePoint’s impact during the pandemic relative to cohorts enrolled 
in CollegePoint before the pandemic. The contrast between remote advising and students’ next 
best alternative for college and financial aid planning support may have decreased during the 
pandemic since students did not have access to in-person supports from March of their senior year 
onward. While overall enrollment declined by approximately ten percent between the pre-COVID- 
19 and COVID-19-affected cohorts, we find that this decline appears to be driven by students who 
would have attended lower-quality institutions. The control group enrollment rate at CollegePoint 
schools overall, and specifically Barron’s 1 and 2 institutions, was virtually the same for cohort 3 
as for cohort 2. The treatment impact for cohort 3 was relatively larger than for cohort 2 (3.2 
percentage points vs. 1.4 percentage points. As we discuss further below, the difference in 
magnitude of program impacts could be due to compositional changes in the experimental sample 


between cohorts. 


The remainder of our paper is structured as follows. In Section II, we provide additional 
background about the CollegePoint initiative. In Section HI, we describe our sample and the data. 


In Section IV, we describe our empirical strategy. We present our results, investigate mechanisms, 


and conduct a cost-benefit analysis in sections V-VII, and conclude with a discussion of the 


importance of our findings and direction for future research and policy in Section VI. 


I. CollegePoint 
In this section, we detail CollegePoint’s recruitment strategy, participating advising organizations, 
and student engagement with advising. Our focus is on students from the high school graduating 


class of 2018 through 2020, which we refer to as Cohorts 1, 2, and 3. 


Recruitment 

CollegePoint works with the College Board the ACT to identify HALIs and HAMIs and promote 
the opportunity to work with a CollegePoint advisor. CollegePoint uses an income cut-off of 
approximately $80,000 and follows Hoxby and Avery (2013) in defining “high-achieving” 
students as those with a GPA of 3.5 or higher; and college entrance exam scores above the 90" 
percentile. Since the SAT and ACT offer their college entrance exam multiple times a year, 
CollegePoint receives several batches of students for recruitment as they become eligible. 
CollegePoint recruits eligible students via direct mail, email, and phone calls. Students outside of 
the universe identified by the ACT and College Board could sign up after verifying the eligibility, 
but very few students did.° Approximately 33 percent of students recruited through the College 


Board signed up.° 


5 4 percent of students from Cohorts 1 and 2 were not in the pool of potentially eligible students identified by the 
ACT and College Board. 

® We base this estimate on data from cohort 2, in which we had access to summary statistics on the full sample 
students who received outreach to participate in CollegePoint. 


Students signed up for CollegePoint on a rolling basis over approximately nine months. Figure 1 
shows that 20 percent of students signed up in March or April of their junior year. About 60 percent 
of students signed up from July through September, and roughly 20 percent of students signed up 
in October or November of their senior year. The take-up date patterns differed across cohorts, but 
the average start dates were similar (see Appendix Figure 1).’ Differences in when students sign 
up largely reflect when they became eligible for the program and CollegePoint’s capacity for new 


students at a given time. 


Advising Organizations 

After signing up, students begin working with an advisor from a consortium of advising 
organizations—College Advising Corps, College Possible, Matriculate, and ScholarMatch. * 
Advisors proactively reach out to their students, but there is no set advising schedule, allowing 


students to interact with their advisor as much or as little as they need. 


The advising organizations fall into one of two categories. College Advising Corps and College 
Possible employ recent college graduates who are full-time employees during a two-year 
AmeriCorps service commitment. Matriculate and ScholarMatch use volunteers who commit to 
working five hours a week.’ The full-time advisors work with 100 to 230 students, while the 
volunteer advisors work with | to 3 students. Given the higher student-to-advisor ratio, College 


Advising Corps and College Possible work with over 75 percent of CollegePoint students. 


7 The averages start date was mid-August for students from Cohort 1, and mid-July for students from Cohorts 2 and 
3. 

8 See https://advisingcorps.org/, http://www.matriculate.org/, https://scholarmatch.org/, https://collegepossible.org/ 
for more information on the advising organizations. 

° Matriculate volunteers are current college students at nine colleges across the country and ScholarMatch volunteers 
span a wide range of ages, professions, but are concentrated in California. 


Experience with CollegePoint Advising 

To contextualize CollegePoint’s remote advising model, we discuss the “dosage” of advising 
students received and their experience with advising. Using advisor-reported interactions and an 
end-of-year survey we administered to students, we find that a nontrivial number of students who 
signed up for CollegePoint had little or no interactions with their advisor.!° Panel A of Figure 2 
shows 26 percent of students who signed up for CollegePoint never interacted with their advisor, 
and 19 percent had 1 or 2 interactions. The distribution of interactions looks similar across cohorts 


(see Appendix Figure 2A-2C). 


CollegePoint’s lower engagement rate relative to some in-person advising models (e.g., Barr and 
Castleman, 2019; Castleman, Deutschlander, and Lohner (2020) could reflect differences in 
student recruitment strategy. CollegePoint proactively reached out through phone, email, and 
postal channels to students pre-screened as likely eligible and had a simple application process. 
In-person models like Bottom Line and College Forward conduct in-person recruitment events in 
schools and have lengthy application processes. Students who complete the application for these 
programs may be positively selected for active and ongoing engagement. In contrast, a greater 
share of CollegePoint students may initially express interest but then decide the program is not one 
they are interested in pursuing. It is also possible that students do not form as strong relationships 
through remote interactions with advisors as they do in person. Finally, the high-achieving students 


that CollegePoint targets may seek out or receive outreach from other college advising supports, 


'0 We describe the survey in more detail in Section III. 


so the “value-add” of CollegePoint advising may be lower than for more academically-mainstream 


student populations served by Bottom Line and College Forward. 


To build intuition about the channels CollegePoint advising may operate through, we examine 
when interactions occurred. Panel B of Figure 2 plots the share of students who engaged by month 
and shows a general decline in engagement throughout the year. The decline in engagement across 
the program year is consistent across cohorts (see Appendix Figure 2D). The decline in 
engagement from 45 percent to 25 percent between the fall and spring suggests that students 
received more advising during the college application process (applications to selective institutions 
are typically submitted by January of senior year in high school) than during the enrollment 


decision process. 


We complement our analysis of the frequency and timing of interactions by examining the topics 
covered during advising interactions. Table 2 presents the share of students who discussed a given 
topic based on advisor-reported data. Column | shows that 51 percent of students discussed the 
college application process, 56 percent discussed applying for financial aid, and 38 percent 
discussed their college enrollment decision. The difference in topics covered reinforces the 
conclusion that CollegePoint’s influence mainly was on where students applied and less so on 


where they chose to enroll (from the choice set of schools to which they were accepted). 


Thus far, the measures of engagement we presented focus on the number of interactions instead of 
how students experienced advising. To more fully examine the student experience, we surveyed 


students from Cohort 1| at the end of the program year. Roughly 70 percent of students responded 


to the survey; respondents were significantly more likely to engage during the program year than 


non-respondents and therefore are more likely to have a favorable view of advising. !! 


As we show in the bottom panel of Table 2, students were most likely to say their advisor helped 
them stay organized by informing them of a task and tracking deadlines. Relative to deciding where 
to apply and enroll, students were more likely to say their advisor was helpful in the financial aid 
process. Specifically, 72 percent and 64 percent of students said their advisor helped with applying 
for aid and understanding award letters, respectively. In comparison, 60 percent and 58 percent of 
students said their advisor helped expand their college list and decide where to enroll, respectively. 
While there were many aspects of advising that students found helpful, students said their advisor 
did not have much influence on where students enrolled. Only 22 percent of students said their 
advisor influenced their college decision, while 66 percent and 40 percent of students said their 


parent and school counselor were influential, respectively. 


CollegePoint evaluation relative to other remote advising evaluations 

To contextualize the contribution of our paper, we present in Table 1 a comparison of CollegePoint 
to other virtual college advising programs that we are aware of having been (1) implemented at 
scale and (2) experimentally evaluated. As we alluded to earlier, our experimental evaluation of 
CollegePoint presents a comprehensive investigation of a remote college advising program that 
(1) is national in scope; (2) examines impacts on both enrollment and persistence in college; and 
(3) investigates multiple mechanisms driving program impacts. Phillips and Reber (2019) conduct 


a multi-cohort RCT of VSOURCE, a remote advising program in Los Angeles. While VSOURCE 


'! For example, respondents had nearly twice as many interactions with their advisor compared to non-respondents. 


provided advising remotely, it did so at the local level. It also recruited students using in-person 
school visits, which would be difficult to replicate in a state or national remote advising model. 
VSOURCE led students to apply to a broader range of four-year institutions but did not affect 
acceptances or enrollment. Gurantz et al. (2019b) evaluate a single-cohort remote advising 
intervention conducted by the College Advising Corps in collaboration with the College Board. 
The student population and advising approach were similar to CollegePoint, and the authors find 
a 2.7 percentage point increase in attendance at higher-quality institutions. But Gurantz et al. 
(2019b) do not investigate persistence impacts and have a minimal investigation of mechanisms 


or student engagement with advising.” 


III. Experimental Design, Data, and Descriptive Statistics 


Experimental Design 


We implemented a randomized trial over three cohorts of CollegePoint students. As students 
signed up for CollegePoint, they completed a brief intake form to verify their eligibility. Based on 
the intake form, 22.5 percent of students were above the income threshold and ineligible for the 
program. We then randomized the remaining students using a 3-to-1 ratio into the CollegePoint 
treatment or control group. '*’!* Since recruitment was a continuous process, we randomized 
students within batches as they signed up. CollegePoint notified treatment students of their advisor 


assignment and told them to expect outreach from their advisor. CollegePoint told students in the 


2 Avery et al. (2020) investigate a single-cohort national text-based advising program implemented by uAspire in 
collaboration with the College Board, but the program was substantially lower in intensity (primarily text-based 
advising; each advisor had a caseload of several thousand students) and served students much lower in the academic 
distribution (mean PSAT scores of 85): The authors estimate a slight negative impact on college enrollment. 


3 CollegePoint contracted with Evaluation & Assessment Solutions for Education (EASE) to conduct the 
randomization and notify advising organizations when students were assigned. 

'4 We switched to a 2-to-1 ratio for part of Cohort | to increase power. The randomization in Cohort 3 began in 
June, so our analysis does not include students who signed up for CollegePoint prior to June 2020. 


control group that they could not be paried with an advisor and were referred to several college 
planning materials.'> Combined across cohorts, our analytic sample consists of 25,696 students 
(19,026 treatmtent and 6,670 control). We excluded the 194 students from our analysis because 
for whom we do not have baseline covariates because they were not recruited through the College 


Board or ACT. 


Data Sources 

Our baseline information on students comes from CollegePoint and its recruitment partners—the 
College Board and ACT. They provided student demographics, high school name, and college 
entrance exam scores at the individual level for cohort 1 and 2 students only and at the cohort 
level for students from cohort 3. CollegePoint collected minimal demographic information when 
students signed up. Students verified they met the income threshold and provided their college 
entrance exam scores and high school names. We matched high schools to the Common Core of 
Data from the National Center for Education Statistics to observe high school characteristics and 


location. 


The advising organizations track their interactions with students and provided the date and 
topic(s) of each interaction. We conducted a post-intervention survey and used advisor-reported 
data to measure students’ engagement and experience with advising. We sent the survey to all 
CollegePoint students from cohort | and a random sample of approximately 1,000 students from 
cohort 2. The survey focused on students’ college decision process and experience with 


advising. NORC at the University of Chicago administered the survey to all students and 


'S Students were told to check out Khan Academy’s resources here: https://www.khanacademy.org/college- 
admissions, as well as College Board Opportunity Scholarships: https://opportunity.collegeboard.org/ and Better 


Make Room: https://www.bettermakeroom.org/up-next/. 


included a $15 incentive to complete the survey. The survey response rate was nearly 70 percent 


and consistent across cohorts. 


We obtained administrative data from the College Board to measure college application 
outcomes. The College Board provided data on SAT score sends, which is often used to 
approximate application behavior but is not a direct measure for where students applied. These 
data are available for 57 percent of the treatment and control groups.'° The majority of students 
from cohorts | and 2 (79 percent) without SAT score sends were recruited through the ACT. 

The lack of coverage for the full sample could potentially impact the generalizability of the result 
to the full sample if CollegePoint was differentially effective for students recruited through the 
SAT, but in section V we provide evidence that the SAT score send results generalize to the full 


sample. 


College enrollment, our primary outcome of interest, comes from the National Student 
Clearinghouse (NSC). The NSC reports a near-universal coverage of postsecondary enrollment, 
allowing us to observe where treatment and control students enrolled. Our analysis focuses on 
the characteristics of the schools where students chose to enroll. We examine enrollment at 
CollegePoint schools since it was an emphasis of the program.'’ We measure college quality 
using college selectivity because colleges do not have a value-added score like in the K-12 
educational setting. Our primary measure of selectivity comes from the Barron’s College Guide, 


which organizes schools into six selectivity tiers. A school’s tier is a function of their acceptance 


'6 The coverage differed across cohorts. SAT score send data were available for 71 percent of students from Cohort 
1, 56 percent of students from Cohort 2, and 41 percent from Cohort 3. 

'” There are approximately 300 schools on the CollegePoint list. Nearly 90 percent of schools with a Barron’s 
ranking of 1 or 2 are on the list, and 36 percent of schools with a Barron’s ranking of 3. 


rate and the GPAs, entrance exam scores, and class ranking of their incoming students. '* We also 
measure college quality using 6-year graduation rates from the Integrated Postsecondary 


Education Data System (IPEDS). 


Baseline Characteristics 

Column | of Table 3 presents descriptive statistics for the students from cohorts | and 2. Consistent 
with the high academic eligibility requirements for CollegePoint, the average student scored above 
the 95" percentile nationally on either the SAT or ACT.'? Approximately 40 percent of the sample 
have a family income below $40,000, and nearly 60 percent have a family income of $40,000- 
$80,000. Over 42 percent of the sample were white, 25 percent were Asian, 19 percent were 
Hispanic, 7 percent were Black, and 5 percent were of some other race or did not provide that 
information. Based on their high school location, 14 percent lived in a rural area, 48 percent lived 
in a suburban area, and 37 percent lived in an urban area. Figure 3 depicts the number of 
CollegePoint students by state and shows that 42 percent of students are from either California, 
Florida, New York, or Texas.7° While not presented in the table, CollegePoint served students 
from over 4,900 high schools in cohort 1, with nearly two-thirds of schools having only one student 
from the experimental sample.”! Appendix Table 2 shows that the demographics did not differ 


substantially across cohorts | and 2. 


'8 The University of Virginia, Stanford University, Oberlin College, and Rice University are examples of Barron’s | 
institutions. The University of Florida, Texas A & M University, Kalamazoo College, and Babson are examples of 
Barron’s 2 institutions. Auburn University, Michigan State University, DePaul University, and George Mason 
University are examples of Barron’s 3 institutions. 

'9 https://collegereadiness.collegeboard.org/pdf/understanding-sat-scores.pdf 

20 Based on state population estimates from the Census, these states contain 33 percent of the population Age 17 and 
younger. Ideally, we would know the state of residence for all eligible students to examine if these states are over 
represented in our sample. 

>! Brooklyn Technical, Stuyvesant, and Bronx Science were outliers and combined for nearly 3.5 percent of the 
cohort | sample. 


To understand who is more likely to sign up for CollegePoint and whether they are positively 
selected, we compare the demographic characteristics and SAT score sends between takers and 
non-takers from a sample of nearly 12,000 eligible students from Cohort 2. Our analysis, which 
we present in Appendix Table 3 shows that, while student groups who traditionally undermatch 
were more likely to sign up for CollegePoint, they were also potentially more interested in 
attending a selective college. CollegePoint students had lower predicted family incomes and were 
significantly more likely to be Hispanic than non-takers. CollegePoint students assigned to the 
control group sent their SAT scores to 1.5 more schools on average and were significantly more 


likely to send the scores to Barron’s 1 schools than non-takers. 


The last column of Table 3 reports p-values for tests of the difference in the characteristics between 
the treatment and control students. Any difference in mean baseline observable and unobservable 
factors related to the outcome would bias our estimated treatment effects. We test whether the 
randomization produced two equivalent groups by examining differences in observable baseline 
characteristics. Column 7 shows no statistically significant differences between the treatment and 
control groups.” The bottom of the table shows an F-test for balance across all variables in Table 
3 and yields a statistically insignificant p-value of 0.93. Given the similarity of the groups on 
observable characteristics, we assume the groups are similar on unobservable characteristics and 


interpret any differences in outcomes as driven by assignment to CollegePoint. 


IV. Empirical Strategy 


>? Appendix Table 2 shows the balance tests by cohort. 


The goal of our study is to evaluate the effect of CollegePoint on where students enroll. We 
estimate the treatment effects by comparing the average outcomes of students assigned to the 
treatment and control group with the following specification: 
(1) Yip = a + BCollegePoint;, + dy + Ein 

where Y;, is an outcome of interest of student i in batch b. The main outcomes are enrollment at 
a CollegePoint school and measures of college quality (e.g., graduation rate, median SAT score 
and instructional spending per student). The variable CollegePoint;yis indicates an individual’s 
assignment to CollegePoint. Since we randomly assigned students to CollegePoint in batches on 
an ongoing basis, we include a batch fixed effect, 6,.To account for the randomization procedure. 
The coefficient of interest 6 represents the causal effect of assignment to CollegePoint. We 
include a limited set of baseline covariates available across all cohorts to increase the precision of 
our impact estimates. Identification in a randomized control trial relies on creating equivalent 
groups at baseline and differ on by only their treatment status. Based on the balance on baseline 
observable characteristics shown in Table 3, we argue that differences in outcomes between the 


treatment and control groups are due to random assignment to CollegePoint. 


V. Results 
Impacts on college applications 
In Panel A of Table 4, we present estimates of CollegePoint’s impact on students’ college 
application behavior (proxied by SAT score sends). As we showed in Section II, most of the 
engagement with advising took place before college applications were due, and college 
applications were the most common topic of discussion between students and advisors. Therefore, 


increasing applications to higher-quality institutions is a likely channel through which 


CollegePoint could influence students’ enrollment options and outcomes. The experimental 
sample applied to numerous colleges (7.9 on average for the control group), and the vast majority 
of the sample (93.7 percent) applied to at least one CollegePoint school. CollegePoint had a slight 
(4 percent) positive effect on the total number of applications and did not affect whether students 
applied to CollegePoint schools. However, assignment to CollegePoint did lead to a 3.0 percentage 
point increase in the share of students who applied to at least one Barron’s | institution (a four 
percent increase relative to the control) and a 1.5 percentage point increase in the share of student’s 


who applied to at least one Barron’s 2 institution (a two percent increase relative to the control). 


Impacts on enrollment and enrollment quality 

In Panels B of Table 4, we present the impacts of CollegePoint on whether and where students 
enroll. CollegePoint did not impact the overall share of students that enrolled in college, though 
overall enrollment rates are very high among this population—86 percent.”? CollegePoint leads to 
a modest increase in the share of students attending CollegePoint schools. Students assigned to 
CollegePoint were 1.3 percentage points more likely to enroll at a CollegePoint school compared 
to a control group mean of 50.1 percent. This impact is primarily driven by increasing enrollment 
at Barron’s | colleges and universities. Students assigned to CollegePoint were 1.1 percentage 
points more likely to attend a Barron’s | institution, a five percent increase relative to the control. 
Increased enrollment at Barron’s 1 corresponded with a decrease in enrollment at less-selective 


institutions (Barron’s 3+ institutions). 


3 This is likely a lower bound for college enrollment, since some students block their records from being shared 
with the NSC while the NSC may not pick up other students’ enrollment if they attend institutions that do not share 
records with the NSC. 


In Panel C of Table 4 and Figure 4, we present the impact of CollegePoint on various measures of 
institutional quality. CollegePoint students attended institutions with slightly higher graduation 
rates than institutions attended by control students (73.1 percent vs. 72.5 percent), but were 
otherwise similar on average to institutions attended by control students. Figure 4 shows the 
distribution of graduation rates for the institutions attended by treatment and control groups. 
Consistent with a small difference in average graduation rate, the distributions are similar. The 
slight increase in the mass in the right tail is consistent with finding impacts on attendance at 
Barron’s | instiutions. We do not find significant differences in CollegePoint impacts across 


student sub-groups.”* 


A unique feature of the CollegePoint RCT is that the first two cohorts preceded the COVID-19 
crisis, while the COVID-19 pandemic began in March, towards the end of the program year for 
students from cohort 3. College enrollment declined substantially among low-income students 
across the country from the high school class of 2020 (Hoover, 2020), and as we show in Table 5, 
we do observe meaningful declines in overall enrollment for cohort 3 students relative to the prior 
cohorts. Whereas roughly 87 percent of the first two CollegePoint cohorts enrolled in college, 80 
percent of the third cohort did so. That being said, this decline appears to be driven by students 
who would have attended lower-quality institutions. The control group enrollment rate at 
CollegePoint schools overall (specifically Barron’s | and 2 institutions) was virtually the same for 
cohort 3 as for cohort 2 (preceding COVID-19). That being said, the control group mean 
enrollment rate at CollegePoint schools declined between the first and subsequent two cohorts: 


52.5 percent of the first cohort control group enrolled at CollegePoint schools, compared with 48.9 


24 Results available upon request 


and 48.2 percent of the second and third cohorts. This decline would have been due to contextual 


factors preceding the COVID-19 crisis. 


Table 5 also shows that the second and third cohorts primarily drive the modest positive effects of 
CollegePoint on attendance at CollegePoint schools for the pooled sample. The treatment impact 
was 1.4 and 3.2 percentage points, respectively. As we show in Appendix Table Al, students in 
the third, COVID-affected cohort had somewhat interactions on average (6.4 vs. 5.8 for cohort 1 
and 5.1 for cohort 2) and more notably, were more likely to discuss their college transition (27.1 
percent of students vs. 23.5 percent for cohort 1 and 18.8 percent for cohort 2). Higher rates of 
interaction and discussion of the college transition in the COVID-19 context could have led to the 
directionally higher impacts of CollegePoint for cohort 3. At the same time, students in cohort 3 
were also more likely to discuss their college list with their advisor (48.9 percent vs. 40.8 percent 
for cohort 1 and 38.8 percent for cohort 2); these discussions would have preceded COVID-19 and 
could also have contributed to the modestly larger impacts for cohort 3. And as we show in 
Appendix Table 2, cohort 3 had a notably larger share of Hispanic students and a lower share of 
white students than the first two cohorts; these compositional changes in the experimental sample 
could also contribute to the modest difference in impacts across cohorts. It is also worth noting 
that the difference in CollegePoint impacts between cohort 2 and 3 is not statistically or practically 
significant to suggest that remote advising’s value relative to the alternative increased substantially 


for high-achieving, low- and moderate-income students during the pandemic. 


In Table 6, we present evidence of CollegePoint’s impact on college persistence. Pooling across 


the first two cohorts, students assigned to CollegePoint were no more likely to be enrolled in 
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college anywhere or at CollegePoint schools in the third semester following high school 
graduation. These general patterns of college persistence seem to hold through five semesters after 
high school among students from the first cohort. Students assigned to CollegePoint were no more 
likely to be enrolled anywhere or at CollegePoint schools or remain continuously enrolled in 


college, but were 1.7 percentage points (8 percent) more likely to enroll at a Barron’s | institution. 


VI. Discussion of Potential Mechanisms 

In this section, we experimentally explore three potential mechanisms underlying the modest 
treatment effects presented above. First, some evidence-based advising programs like College 
Forward start working with their students earlier than CollegePoint; this longer duration of 
engagement could be important for program efficacy. We examine the effect of a longer program 
year by randomizing when students began advising. Second, the organizations delivering 
CollegePoint advising differ along several dimensions (See Section II), which could influence 
program efficacy. We use the random assignment of students to advising organizations to estimate 
differences in effects by organization. Third, programs like Bottom Line and College Forward 
work with students who are on the margin of going to four-year institutions, while CollegePoint 
works with students who, in the absence of advising, primarily attend at least moderately selective 
colleges and universities. College advising programs may be effective for more academically- 
mainstream students but less so for high-achieving students who have access to more college and 
financial aid planning alternatives. To understand if students with a different academic 
background would benefit from advising, CollegePoint conducted a pilot with students somewhat 


lower in the academic distribution. 
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Mechanism 1: Duration of Advising 

Could CollegePoint be more effective if it started working with students earlier? Figure 1 shows 
that only 20 percent of students signed up for CollegePoint before May of their junior year of high 
school and that 60 percent signed up after the start of their senior year. Compared to programs 
like College Forward, which starts working with students at the beginning of their junior year in 
high school, CollegePoint students spent less time on average enrolled in the program. A longer 
program could allow advisors to form deeper relationships and cover more college and financial 


aid topics with students. 


To examine the potential benefits of a longer program year, we randomized when students started 
advising. We conducted this RCT with the high school class of 2017, which preceded the three 
cohorts we include in the overall RCT of CollegePoint that we report on in this paper. Among 
students who signed up and were eligible for the program in the spring of their junior year of high 
school, we randomized students to one of two groups. Early starters started receiving outreach 
from their advisor in April, and late starters started receiving outreach from their advisor in 
September. Early starters received three additional months of advising because no advising occurs 
in July or August. The experiment consisted of 3,101 students randomized in equal proportion 
between groups. Appendix Table 4 shows the balance of baseline characteristics between the 


groups. 


Beginning advising in April rather than September had significant impacts on levels of engagement 


but no effect on college enrollment outcomes. Panel A of Table 7 shows that early starters on 


average had 8.2 interactions with their advisor compared to 5.5 among late starters. Early starters 
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were also significantly more likely ever to discuss a variety of topics with their advisor. For 
instance, 42.6 percent of early starters reported discussing their college list with their advisor, 
compared with 29 percent of late starters. However, most of these differences in engagement 
occurred three months before the late starters began advising. Appendix Figure 3 shows that the 
monthly interaction rate did not differ from September through the end of the program year. The 
figure is consistent with our finding in Table 7 that early starters were no more likely to discuss 
their college enrollment decision with their advisor. Panel B and C show that starting earlier did 
not affect either college applications or college enrollment. While a substantially longer extension 
of the CollegePoint program model (e.g., to the start of junior year) could lead to larger impacts. 
The “early vs. late start’ RCT results show that extending the duration of remote advising by 


several months (at least for high-achieving students) was insufficient to generate larger effects. 


Mechanism 2: Variation in remote advising efficacy across organizations 

The organizations providing advising through CollegePoint differ along important dimensions. 
Two of the organizations are large non-profit organizations that have operated for well over a 
decade and with numerous sites across the country where they offer in-person college advising to 
students. These organizations use full-time, recent college graduates as CollegePoint advisors and 
had larger caseloads (125-250 students per advisor). The other two organizations are 
comparatively smaller and newer non-profit organizations. One of the organizations uses current 
college students at selective institutions as mentors; the other relies on recent college graduates 
and adult volunteers. Both of these latter organizations’ advisors have small caseloads (1-3 
students per advisor). While we are not able to isolate the particular aspect of an organization that 


might contribute to larger or smaller impacts of remote advising, we nonetheless might expect 
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these differences in organizational approach to contribute to different magnitudes of program 


impact. 


In Table 8 we present CollegePoint impacts separately by organization (pooling across all three 
experimental cohorts). Three of the four organizations increased the share of treated students that 
applied to Barron’s 1 institutions, with impacts raning from 2.6 to 5.4 percentage points), though 
none of the organizations led treated students to apply to CollegePoint schools at significantly 
higher rates. One of the longer-standing organizations with full-time advisors and larger caseloads 
generated a modest increase (1.8 percentage points) in CollegePoint attendance, while the 
organization that uses current college students as peer mentors generated a 4.4 percentage point 
increase in CollegePoint attendance. This latter effect was driven entirely by increasing the share 


of students who attended Barron’s 1 institutions. 


The results by organization do not solely reflect differences in organizational effectiveness. As we 
show in Appendix Table 5, each organization’s overall composition of students differs somewhat, 
owing to differences in when organizations enrolled students during the CollegePoint program 
year. For instance, organizations 3 and 4 (the newer organizations with smaller caseloads staffed 
by peer mentors (organization 3) or volunteers (organization 4) worked with larger shares of 


students from families making less than $40,000 per year than organizations | and 2. 


The differences in enrollment impacts by organization did not manifest themselves into differences 


in students’ CollegePoint advising experience. As we show in Appendix Table 6, with the 


exception of organization 1, students assigned to the other organizations responded similarly to 
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questions about their advisor relationship and the helpfulness of advising. The main exception is 
with respect to influence in making a college enrollment decision.”> Panel C shows that students 
assigned to organization 3 were over twice as likely to say their advisor influenced their college 
decision compared to students assigned to all other organizations. We unfortunately have limited 
data to assess how advisors from organization 3 had more influence in the college enrollment 
decision. Panels A and B show that the difference in influence is not due to a closer relationship 


or more helpful advising. 


Given that organizations varied in the composition of students they served, we cannot disentangle 
for the overall CollegePoint population whether these differences in organization-level impacts are 
due to some aspect of the organizational model or due to differences in the students they worked 
with. In Table 9 we present impact estimates for a subset of the CollegePoint population in which 
students (N=4,013) were randomly assigned between organization 2 and organization 3. Among 
this subset both organizations generated a positive impact on enrollment at CollegePoint schools: 
3.2 percentage points for organization 2; 4.8 percentage points for organization 3. Organization 
3’s impacts for this subset were again driven entirely by increasing enrollment at Barron’s 1 
institutions. The last column of Table 9 shows that some of the relatively larger impacts for 
organization 3 are statistically different from the impacts of organization 2, which provide 
suggestive evidence that some combination of advisors being current students at selective 


institutions and having smaller caseloads may have contributed to greater CollegePoint efficacy. 


Mechanism 3: Academic profile of students in college advising programs 


25 Organization 2 seems particularly strong in guiding students the financial aid process. 
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To investigate whether remote advising would be more effective for students with lower observed 
baseline academic achievement, CollegePoint conducted small pilots with cohorts 2 and 3 with 
students slightly lower in the academic distribution (at or below the 89th percentile of SAT scores) 
compared to their high-achieving peers. Borrowing from a College Board designation, 
CollegePoint referred to these students as “on-track.” While students at this point in the academic 
distribution do not allow us to compare CollegePoint’s student population to effective in-person 
models directly. The pilot nonetheless provides some evidence of whether CollegePoint’s impacts 


are differentially effective for students at a modestly different point in the academic distribution. 


Across the two cohorts, we randomized 3,394 on-track students to receive CollegePoint advising 
or to a business-as-usual control group. Programmatically, the only aspect of CollegePoint that 
differed for on-track students is when they started working with their advisor. On average, on- 
track students began two months after their high-achieving peers. Appendix Table 4 shows that on 
average, on-track students scored at the 89" percentile on the SAT, six percentiles lower than 
students in our main analytic sample. Compared to the main analytic sample, on-track students 
are more likely to be from low-income families and first-generation college students. Specifically, 
69 percent are first-generation college students, which is still 12 percentage points lower than the 
population of students working with Bottom Line (see Table 1). On-track students were also more 
likely to be black or Hispanic and less likely to be white or Asian. By serving students with slightly 
lower SAT scores, CollegePoint recruited students based on demographics who could have less 


information about the college search process and could potentially benefit more from advising. 
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Overall, Table 10 shows little evidence that on-track students benefitted CollegePoint. However, 
we observe modest shifts in the quality of institution that students apply to and attend by virtue of 
assignment in CollegePoint. Consistent with the differences in baseline academic achievement, 
on-track students were significantly less likely than high-achieving students to apply to and enroll 
at the most selective schools. For instance, 74.8 percent of the high-achieving sample applied to a 
Barron’s 1 institution compared to 61.7 percent of on-track students. And, 19.9 percent of the high- 
achieving sample enrolled at a Barron’s | institution compared to 9.5 percent of on-track students. 
In Panel B, we show that on-track students in CollegePoint were no more likely to attend 
CollegePoint schools. Assignment to CollegePoint did increase on-track student attendance at 
Barron’s 3 institutions by 2.7 percentage points (a 13 percent increase relative to the control), but 
this increase was offset by slight though non-significant declines in enrollment at Barron’s 1 and 
2 institutions. Panel C shows that this shift in composition did not result in significant mean 
differences in the quality of institutions treated on-track students attended. However, it is possible 


advisors and students were optimizing other preferences or considerations. 


The on-track RCT suggests that remote advising is not likely to increase substantially in efficacy 
through modest changes to the academic profile of students recruited for the program. Moreover, 
the on-track student RCT does not directly address whether remote advising could generate larger 


effects for student populations on the margin of four-year attendance. 


VII. Conclusion 
Our paper demonstrates that remote college advising for high-achieving low- and moderate- 
income students leas to modest increases in the share of students that enroll and persist at higher- 


quality institutions. Our mechanism experiments show that, at least for the high-achieving student 
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population, the magnitude of remote advising programs’ impact is unlikely to increase markedly 
by modest expansions to the duration of advising or to the academic eligibility thresholds for 
participating students. We moreover show that, despite that vast majority of the cohort 3 
experimental sample not having access to in-person advising at their school or community from 
March of their senior year onward during the COVID-19 pandemic, the value-add of remote 


advising did not increase substantially. 


CollegePoint cost $562 per student served. A primary reason the costs are substantially lower than 
evidence-based in-person college advising models is that CollegePoint serves students for one year 
(or in some cases slightly more than a year) whereas intensive programs like Bottom Line and 
College Forward serve students for 5-6 years. To estimate CollegePoint’s cost per student 
impacted by the program, we use enrollment at a CollegePoint school as a measure of enrollment 
quality. CollegePoint’s cost per student induced to enroll at a higher quality institution is $43,275. 
As we report above, we find that the this initial impact on enrollment at CollegePoint school fades 
out by students’ second year out of high school—though we do find persistent impacts (~1.5pp) 
of attendance at Barron’s | institutions. We do not know of prior empirical estimates of the 
economic return to attending a slightly higher quality institution, when the mean outcome for the 
control group is still attendance at institutions with quite high graduation rates. Even in the liberal 
scenario where there are modest gains to attending slightly higher quality institutions, it would 
likely be several years before the net present value of these earnings gains exceeded the costs of 


inducing students to attend Barron’s | colleges and universities. 
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Given the informational and assistance barriers that low-income students in many communities 
face, there is still value in investigating remote college advising models, particularly for more 
academically-mainstream populations that have benefited substantially from community-based 


advising. 
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Panel A: Program Features 
Virtual (Y/N) 
Start Date 
End Date 
Geographic Focus 
Engagement with 
Advising 
Advisor Information 


Student Academic 
Eligibility 


CollegePoint 


Y 
Fall Senior Year 
End of HS 
National 


73.2% interacted; 5.7 
interactions on avg 


3.5 GPA 


Panel B: Demographic Characteristics of Sample 


% Parent w Bachelor 
% White 

% Hispanic 

% Black 

% Asian 


Panel C: Impact 
Enrollment Outcome 


Sources: Phillips and Reber (2019), Gurantz et al. (2019b), Castleman, Deutschlander, and Lohner, (2020), Barr and Castleman (forthcoming) 


36 
35.8 
17 
6.2 
22.2 


1.4pp (2.8%) increase in 
CP school enroll 


1.3pp (6.4%) increase in 
Barron's | attendance 


Table 1 
Comparison of Advising Organizations 


College Forward 


N 
Jan. Junior Year 
Through college 


Austin and Houston 


twice-weekly class, and 1- 


on-| meetings 


Top 60% of HS class 


25 non first gen 
6.8 
66.6 
13.9 
3.6 


CAC/College Board 


Y 
Spring Junior Year 
End of HS 


National 


44% ever interacted.' 


Full-time, AmeriCorps members 
Top 10% SAT/ACT, and 


Top 10% SAT, and 3.5 
GPA 


39.5 
38.4 
17.6 
5.2 
32.7 


7.5pp (12.9%) increase in 2.7pp (5.4%) increase in 


college enrollment 


CP school enroll 


1.7pp increase in Barron's 
1-3 attendance 


V Source 


Y 
March Junior Year 
End of HS 
Los Angeles Area 


11 written interactions, 2 


voice-to-voice 
College undergrads 


No requirement; 75% 
above 3.0 GPA 


14.8 
4.6 
ISA 
6 
10.9 


No effects on enrollment 


Bottom Line 


N 
Summer Junior Year 
Max six years post HS 
Boston and NYC 


hourly every 3-4 weeks; 
avg of 13 


Full-time, near-peer 


2.5 GPA 


19 (81% first gen) 


32.5 
30.2 
24.6 


Tpp (8.5%) increase in 
college enrollment 


Notes: (1) Unlike CollegePoint, the CAC/CB study identified eligible students and assigned them to a treatment group before they had taken up the offer of virtual 
advising; CollegePoint students take up the offer and then are randomized, hence higher engagement rates. 
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Table 2 
Intervention Dosage: Advising Usage and Experience 


Mean N 
(1) (2) 
Panel A: Advisor Records 
% Ever Interacted 74.6 23990 
Avg. Number of Interactions 5.7 23990 
% Discussed College List 40.6 23990 
% Discussed College Applications 51.4 23990 
% Discussed Applying for Financial Aid 56.3 23990 
% Discussed College Decision 37.7 23990 
% Discussed College Transition 20.0 23990 
Panel B: Student Survey 
% Said Advisor Helped Track Deadlines 74.3 5201 
% Said Advisor Encouraged Expanding College List 60.3 5207 
% Said Advisor Helped Complete Applications 61.6 5188 
% Said Advisor Helped with Applying for Aid 72.4 5206 
% Said Advisor Helped Manage Stress 66.0 5206 
% Said Advisor Helped with Understanding Aid 63.8 5197 
% Said Advisor Helped with College Decision 57.7 5201 
% Said Advisor Influenced College Decision 22.0 5086 


Notes: The table shows usage and experience of CollegePoint advising for students in the treatment group 
based on advisor records and student self-reports from a post-intervention survey. Panel A includes all 
students assigned to the treatment group. Panel B includes the students from cohorts 1 and 2 who responded 
to the survey. See section III for more details about the survey. 
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Table 3 
Summary Statistics and Balance Tests for Students from Cohorts 1 and 2 


Pooled Control Treatment P-Value for 
Mean N Mean N Mean N T-C Diff 

(1) (2) (3) (4) (5) (6) (7) 
Percentile College Entrance Exam Score 95.4 21073 95.3 5531 95.5 15542 0.271 
% Family Income < $40,000 40.3 21073 40.4 5531 40.3 15542 0.309 
% Family Income $40,000- $80,000 58.2 21073 58.1 5531 58.3 15542 0.351 
% Family Income > $80,000 1.5 21073 1.5 5531 1.5 15542 0.760 
% Asian 25.4 20052 25.2 5265 25.5 14787 0.974 
% Black 6.9 20052 74 5265 6.8 14787 0.244 
% Hispanic 19.2 20052 18.7 5265 19.4 14787 0.452 
% White 42.8 20052 43.0 5265 42.8 14787 0.791 
% Other Race 5.6 20052 537 5265 535 14787 0.649 
% Parents without a BA 52:5 19589 52.0 5132 52.7 14457 0.571 
% Female 54.9 20382 55.3 5353 54.8 15029 0.676 
% Urban High School 37.2 20793 37.7 5460 37.0 15333 0.304 
% Suburban High School 48.3 20793 48.1 5460 48.4 15333 0.610 
% Rural High School 14.4 20793 14.2 5460 14.5 15333 0.491 
% Public High School 77.2 20671 76.8 5423 77.3 15248 0.374 
% Private High School 74 20671 73 5423 74 15248 0.595 
% Charter High School 4.2 20671 4.5 5423 4.1 15248 0.131 
% Magnet High School 11.5 20671 11.7 5423 11.4 15248 0.501 
% Low-Supply CP School State 13.3 21073 13.1 5531 13.3 15542 0.700 
% Mid-Supply CP School State 23.4 21073 23.1 5531 23.5 15542 0.521 
% High-Supply CP School State 63.3 21073 63.8 5531 63.2 15542 0.394 
School Size 1673 20677 1685 5426 1668 15251 0.278 
HS Student:Counselor Ratio 291 19142 294 5029 291 14113 0.139 
% FRL 42.2 18782 42.3 4941 42.1 13841 0.454 
HS Per Pupil Expenditure 15587 19143 15688 5030 15551 14113 0.193 

F-Test F-Stat P-Value N 


0.424 0.968 21073 


Notes: This table presents baseline summary statistics for the students in the experimental sample from cohorts | and 2. 
Due to changes with a data provider, we lack detailed student-level data for students from cohort 3. We present cohort- 
level demographics in Appendix Table 2. We present means and counts for the full sample, and for the treatment and 
control groups separately. In Column 7, we show the p-value for a test of the difference between treatment and control 
group means, estimated by regressing the relevant baseline variable on the treatment group indicator and a randomization 
batch fixed effect. We report a joint balance test by regressing treatment status on all baseline variables in the table 
excluding high school characteristics. The bottom of the table contains the resulting F-statistic and p-value. All regressions 
use robust standard errors.* p<0.1, ** p<0.05, *** p<0.01 
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Table 4 


Effect on College Applications & Enrollment 


Control 
Mean 
(1) 
Panel A: Score Send Outcomes 
Total Applications 7.9 
% Applied to CollegePoint School 93.7 
% Applied to Barron's 1 74.7 
% Applied to Barron's 2 73.2 
% Applied to Barron's 3 57.0 
% Applied to Barron's 4+ 62.1 
Observations: 14658 
Panel B: Enrollment Outcomes 
% Enrolled Anywhere 85.9 
% Enrolled at CollegePoint School 50.1 
% Enrolled at Barron's | 20.1 
% Enrolled at Barron's 2 22.1 
% Enrolled at Barron's 3 19.1 
% Enrolled at Barron's 4+ 24.6 
Observations: 25696 
Panel C: College Characteristics 
SAT midpoint “tile 88.5 
Instructional expenditures per FTE ($1,000s) 223 
Graduation rate (%) 72.5 
Net price: Family Income 30-48k 12380 
Net price: Family Income 48-75k 16075 
Observations: 22004 


Treatment 
Effect 


(2) 


0.4 
0.3 
3.0 
1.5 
-0.4 
-0.9 


-0.3 
1.3 
1.1 
0.1 

-0.5 

-0.9 


0.3 
0.2 
0.6 
-78.69 
7.38 


SE 
GB) 


(0.163) 
(0.445) 
(0.786) 
(0.811) 
(0.928) 
(0.913) 


(0.499) 
(0.689) 
(0.570) 
(0.584) 
(0.548) 
(0.598) 


(0.174) 
(0.327) 
(0.280) 
(93.318) 
(97.058) 


P-value 
(4) 


0.031** 
0.519 

0.000*** 
0.060* 
0.658 
0.350 


0.513 
0.069* 
0.065* 
0.917 
0.317 
0.132 


0.086* 
0.541 

0.029** 
0.399 
0.939 


Notes: This table reports the treatment effects of CollegePoint on students’ application and initial college 
enrollment choices. Panel A presents the treatment effects on college applications as measured by SAT score 
sends. Panel B presents the treatment effects on whether and where students enroll. Panel C presents the 


treatment effects on a variety of college characteristics and is conditional on enrollment. The college 


characteristics come from IPEDS. We calculate treatment effects by regressing the relevant outcome variable on 
the treatment group indicator, a subset of covariates in Table 3, and a randomization batch fixed effect. Robust 
Standard errors are reported in parentheses. *<0.1, ** p<0.05, *** p<0.01 
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Table 5 
Effect on College Applications and Enrollment by Cohort 


Cohort 1 Cohort 2 Cohort 3 
Ctrl Treat Ctrl Treat Ctrl Treat 
Mean Effect mE E-val Mean Effect ae ay Mean Effect RE Peval 


Gd) @) (3) (4) (5) (6) 7) (8) (9) 0) Gl) 2) 


Panel A: Score Send Outcomes 


Total Applications 7A 0.2 (0.131) 0.080* 8.5 0.6 (0.337) 0.064* 8.8 -0.1 (0.381) 0.754 
% Applied to CollegePoint School 93.9 -0.2 (0.658) 0.768 92.7 1.1 (0.712) 0.140 96.5 -04 (1.076) 0.738 
% Applied to Barron's 1 73.2 4.1 (1.167) 0.000*** = 75.1 2.5 (1.212) 0.037** 78.8 0.6 (2.220) 0.775 
% Applied to Barron's 2 70.7 18 (1.234) 0.154 75.2 16 (1.221) 0.177 76.2 0.1 (2.294) 0.954 
% Applied to Barron's 3 53.5 0.4 (1.384) 0.796 60.9 -1.3 (1.414) 0.351 57.2 -0.2 (2.687) 0.945 
% Applied to Barron's 4+ 59.7 -3.2.— (1.378) 0.019** 63.9 1.7 (1.380) 0.230 65.3 -l.1 (2.599) 0.680 
Observations: 6430 6350 1878 

Panel B: Enrollment Outcomes 
% Enrolled Anywhere 87.2 -0.6 (0.780) 0.457 87.1 -0.8 (0.728) 0.281 79.6 14 (1.347) 0.313 
% Enrolled at CollegePoint School 52.4 0.3 (1.121) 0.814 48.9 14 (1.024) 0.174 47.5 3.2 (1.672) 0.057* 
% Enrolled at Barron's 1 20.7 1.1 (0.945) 0.230 19.8 10 (0.841) 0.215 19.4 1.0 (1.365) 0.460 
% Enrolled at Barron's 2 22.8 0.4 (0.972) 0.677 21.7. -0.3 (0.859) 0.687 21.4 0.4 (1.391) 0.752 
% Enrolled at Barron's 3 19.2 -0.3 (0.895) 0.745 20.4 -1.1 (0.834) 0.204 15.5 0.2 (1.232) 0.897 
% Enrolled at Barron's 4+ 24.5  -1.8 (0.958) 0.054* 25.2 -04 (0.907) 0.641 23.3 -0.2 (1.422) 0.862 
Observations: 9249 11824 4623 

Panel C: College Characteristics 
SAT midpoint “tile 88.7 0.3 (0.280) 0.243 88.4 0.2 (0.255) 0.338 88.4 0.4 (0.458) 0.355 
Instructional expenditures per FTE ($1,000s\ 22.4 0.2 (0.521) 0.734 22.0 0.1 (0.481) 0.882 22.7 0.7 (0.853) 0.421 
Graduation rate (%) 73.3 0.4 (0.447) 0.416 72.0 0.4 (0.414) 0.291 72.0 1.8 (0.720) 0.014** 
Net price: Family Income 30-48k 12418 -77 (152.680) 0.612 12553, -215 (137.778) 0.118 11802 279 (228.218) 0.222 
Net price: Family Income 48-75k 16131 -3 (158.195) 0.987 16218 -113 (143.283) 0.429 15529 350 (239.536) 0.144 
Observations: 8043 10228 3733 


Notes: This table reports the impact estimates in Table 4 separately by cohort. Robust Standard errors are reported in parentheses. See Table 4 for 
further details. *<0.1, ** p<0.05, *** p<0.01 
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Table 6 
Effect on College Persistence 


Cohorts | and 2 Cohort | Cohort 2 
Ctrl Treat Ctrl Treat Ctrl Treat 
Mean Effect RE Eyal Mean Effect ae Eeval Mean Effect ae Fval 


Gd) @ (3) (4) ) (6) (7) (8) (9) G0) Gl) (2) 


Panel A: Term 3 Outcomes 


% Enrolled Anywhere 85.0 -0.7 (0.565) 0.224 85.5 0.9 (0.826) 0.282 84.6 -0.5 (0.774) 0.490 
% Enrolled at CollegePoint School 49.6 0.7 (0.740) 0.349 51.8 -0.1 (1.098) 0.903 47.6 14 (1.001) 0.149 
% Enrolled at Barron's | 20.2. 0.9 (0.608) 0.123 20.7 13 (0.919) 0.152 19.8 0.6 (0.810) 0.454 
% Enrolled at Barron's 2 21.4 0.2 (0.633) 0.702 22.3 0.1 (0.958) 0.898 20.7 04 (0.845) 0.640 
% Enrolled at Barron's 3 19.0 -0.8 (0.596) 0.173 18.6 -0.8 (0.873) 0.336 19.3 -0.8 (0.816) 0.307 
% Enrolled at Barron's 4+ 244 -1.1 (0.644) 0.098* 23.9 -1.5 (0.937) 0.112 24.8 -0.7 (0.885) 0.414 
% Continuously Enrolled 82.0 -0.6 (0.605) 0.322 82.7 -1.3 (0.885) 0.148 81.3 -0.1 (0.827) 0.931 


% Continuously Enrolled at CP School 47.4 0.6 (0.742) 0.444 49.7 -0.2. (1.103) 0.851 45.4 1.2 (1.004) 0.214 
Panel B: Term 5 Outcomes 


% Enrolled Anywhere 82.6 -0.8 (0.883) 0.348 
% Enrolled at CollegePoint School 50.3 0.1 (1.103) 0.918 
% Enrolled at Barron's 1 19.7 1.7 (0.907) 0.068* 
% Enrolled at Barron's 2 22.1 -0.3 (0.951) 0.726 
% Enrolled at Barron's 3 18.3 -0.8 (0.869) 0.377 
% Enrolled at Barron's 4+ 22.4 -14 (0.917) 0.132 
% Continuously Enrolled 774 -1.2 (0.969) 0.204 
% Continuously Enrolled at CP School 46.7 -0.1 (1.107) 0.920 
Observations: 21073 9249 11824 


Notes: This table reports the treatment effects of CollegePoint on persistence in college. Panel A reports the treatment effects on whether and where 
students are enrolled three semesters after graduating high school. Panel B reports the same effects but looking out five semester after high school 
graduation. Based on when students graduated high school, third semester outcomes are available for cohorts 1 and 2, and fifth semester outcomes are 
available for cohort 1. We calculate treatment effects by regressing the relevant outcome variable on the treatment group indicator, a subset of covariates 
in Table 3, and a randomization batch fixed effect. Robust Standard errors are reported in parentheses. *<0.1, ** p<0.05, *** p<0.01 
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Table 7 
Effects of Starting CollegePoint Early 
Late Start Early Start 


Mean Effect Pe ae 
(1) (2) 3) (4) 
Panel A: Engagement in Advising 
Number of Interactions 5.5 2.7 (0.283) 0.000*** 
% Ever Interacted 79.2 9.3 (1.313) 0.000*** 
% Discussed Academic Prep TD 14.0 (1.239) 0.000*** 
% Discussed College Applications 34.1 11.8 (1.746) 0.000*** 
% Discussed College List 29.0 13.6 (1.704)  0.000*** 
% Discussed Financial Aid Applications 50.9 9.0 (1.774) 0.000*** 
% Discussed College Enrollment Decision 36.9 1.8 (1.740) 0.296 
Observations: 3101 
Panel B: Score Send Outcomes 
Total Score Sends 7.7 -0.1 (0.195) 0.533 
Score Sends to CollegePoint Schools 6.3 -0.1 (0.190) 0.492 
Score Sends to Barron's | Schools 4.2 -0.1 (0.165) 0.758 
Score Sends to Barron's 2 Schools 1.7 0.0 (0.063) 0.652 
Score Sends to Barron's 3 Schools 0.8 -0.1 (0.042) 0.222 
Score Sends to Barron's >3 Schools 1.0 0.0 (0.047) 0.860 
Observations: 2690 
Panel C: Enrollment Outcomes 
% Enrolled in College 86.0 -1.1 (1.269) 0.374 
% Enrolled at CollegePoint School 58.1 -1.9 (1.778) 0.289 
% Enrolled at Barron's 1 School 27.8 -0.5 (1.603) 0.743 
% Enrolled at Barron's 2 School 24.2 1.1 (1.545) 0.486 
% Enrolled at Barron's 3 School 16.7 -2.3 (1.296)  0.078* 
% Enrolled at Barron's >3 School 17.3 0.6 (1.367) 0.657 
Observations: 3101 


Notes: This table reports the treatment effects of starting CollegePoint in the spring of junior year ("early starters") 
versus the fall of senior year ("late starters"). Panel A presents the treatment effects on engagement with advising. 
Panel B presents the treatment effects on college applications as measured by SAT score sends. Panel C presents 
the treatment effects on whether and where students enroll. The experiment occurred within the cohort prior to the 
main experimental sample. We calculate treatment effects of starting early by regressing the relevant outcome 
variable on the early starter group indicator, a subset of covariates in Table 3 and a randomization batch fixed 
effect. Robust Standard errors are reported in parentheses. *<0.1, ** p<0.05, *** p<0.01 
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Table 8 
CollegePoint Impacts by Advising Organization 


Org 1 Org 2 Org 3 Org 4 
Control Treat Treat Treat Treat 
Mean Effect Be Effect Re Effect oe Effect e 
(1) (2) (3) (4) (5) (6) (7) (8) (9) 

Panel A: Score Send Outcomes 
Total Applications 7.9 0.2 (0.223) 0.3 (0.205) 0.6 (0.292)** 0.7 (0.345)* 
% Applied to CollegePoint School 93.7 0.2 (0.609) 0.1 (0.560) 0.6 (0.796) 0.8 (0.942) 
% Applied to Barron's 1 74.7 2.6 (1.076)** 3.0 (0.990)*** 5.4 (1.406)*** 0.4 (1.663) 
% Applied to Barron's 2 73.2 1.7 (1.111) 1.4 (1.022) 1.7 (1.452) 1.6 (1.718) 
% Applied to Barron's 3 57.0 -0.3 (1.271) -0.8 (1.169) -1.3 (1.660) 2.2 (1.964) 
% Applied to Barron's 4+ 62.1 -0.3 (1.250) -0.4 (1.150) -2.6 (1.634) -1.7 (1.932) 
Observations: 14658 

Panel B: Enrollment Outcomes 
% Enrolled Anywhere 85.9 -1.0 (0.669) -0.1 (0.620) 0.8 (0.948) -0.6 (1.176) 
% Enrolled at CollegePoint School 50.1 -0.3 (0.924) 1.8 (0.856)** 4.4 (1.309)*** -0.5 (1.624) 
% Enrolled at Barron's | 20.1 0.2 (0.765) 0.6 (0.709) 4.9 (1.084)*** 0.9 (1.345) 
% Enrolled at Barron's 2 22.1 -0.5 (0.783) 1.1 (0.726) -2.0 (1.110)* -0.2 (1.377) 
% Enrolled at Barron's 3 19.1 -0.6 (0.735) -0.6 (0.681) 0.5 (1.041) -1.5 (1.292) 
% Enrolled at Barron's 4+ 24.6 -0.1 (0.802) -1.2 (0.743)* -2.6 (1.136)** 0.3 (1.410) 
Observations: 25696 

Panel C: College Characteristics 
SAT midpoint “tile 88.5 0.0 (0.236) 0.3 (0.216) 0.8 (0.330)** 0.1 (0.414) 
Instructional expenditures per FTE ($1,000s) 22.3 -0.3 (0.441) -0.1 (0.404) 2.1 (0.622)*** 0.6 (0.780) 
Graduation rate (%) 72.5 0.0 (0.378) 0.7 (0.346)* 2.0 (0.533)*** 0.3 (0.668) 
Net price: Family Income 30-48k 12380 -127. = (125.909) 17 (115.358) -356 = (177.518)** 25 (222.550) 
Net price: Family Income 48-75k 16075 -50 (130.957) 97 (119.984) -213 = (184.576) 88 (231.426) 
Observations: 22004 


Notes: This table reports the impact estimates in Table 4 by advising organization. Each row contains the output from a single regression. Instead of 
including a single treatment group indicator, we included indicators for assignment to the four advising organizations. Robust Standard errors are reported in 
parentheses. See Table 4 for further details. *<0.1, ** p<0.05, *** p<0.01 
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Table 9 
Differences in Organizational Effectiveness 


. ; 7 Org 2 - Org 3 F-tast:p- 
ontro reat reat 

Mean Effect 5 Effect Be ae 

Q) (2) (3) (4) (5) (6) 

% Enrolled Anywhere 86.1 0.6 (1.335) 0.8 (1.423) 0.907 
% Enrolled at CollegePoint School 51.8 32 (1.903)* 4.8 (2.029)** 0.405 
% Enrolled at Barron's | 22.5 1.4 (1.667) 48 (1.776)*** 0.042 
% Enrolled at Barron's 2 22.5 2.1 (1.626) -1.0 (1.733) 0.065 
% Enrolled at Barron's 3 17.5 -0.9 (1.443) 0.0 (1.538) 0.563 
% Enrolled at Barron's 4+ 23.6 -2.0 (1.576) -3.1 (1.679)* 0.496 


Observations: 4013 


Notes: This table reports the impact estimates in Table 4 for the subset of students who were randomized between 
organizations 2 and 3. Each row contains the output from a single regression. Instead of including a single 
treatment group indicator, we included indicators for assignment to the two advising organizations. The last 
column reports the p-value for the joint test of equality between the organizations effects. Robust Standard errors 
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Table 10 
Effect on College Applications & Enrollment for On-Track Students 


Control Mean Picanent SE P-value 
Effect 
CQ) (2) (3) (4) 

Panel A: Score Send Outcomes 
Total Applications 7.9 -0.1 (0.291) 0.782 
% Applied to CollegePoint School 89.8 0.2 (1.381) 0.892 
% Applied to Barron's 1 61.7 0.4 (2.237) 0.850 
% Applied to Barron's 2 73.4 -4.5 (2.082) 0.031** 
% Applied to Barron's 3 65.8 6.4 (2.132) 0.003*** 
% Applied to Barron's 4+ 77.9 -1.3 (1.937) 0.496 
Observations: 1877 

Panel B: Enrollment Outcomes 
% Enrolled Anywhere 82.8 0.6 (1.283) 0.640 
% Enrolled at CollegePoint School 36.7 0.5 (1.615) 0.764 
% Enrolled at Barron's 1 9.5 -0.7 (0.984) 0.492 
% Enrolled at Barron's 2 18.0 -1.0 (1.286) 0.449 
% Enrolled at Barron's 3 20.6 2.7 (1.412) 0.055* 
% Enrolled at Barron's 4+ 34.7 -0.5 (1.620) 0.777 
Observations: 3394 

Panel C: College Characteristics 
SAT midpoint “tile 84.1 -0.3 (0.500) 0.518 
Instructional expenditures per FTE ($1,000s) 16.0 -0.3 (0.463) 0.457 
Graduation rate (%) 65.7 0.1 (0.743) 0.890 
Net price: Family Income 30-48k 13024 -310 (227.124) 0.173 
Net price: Family Income 48-75k 16523 -297 (231.084) 0.198 
Observations: 2821 


Notes: This table reports the impact estimates in Table 4 from a pilot of CollegePoint with on-track students. 
Robust Standard errors are reported in parentheses. See Table 4 for further details. *<0.1, ** p<0.05, *** p<0.01 
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Figure 1: CollegePoint Start Date 
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Notes: The line plot shows the cumulative share of students who signed up for CollegePoint by week. The vertical 
dashed line represents the average start date. 
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Figure 2: Interactions with Advisor 


A) Distribution of Interactions 
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B) Interaction Rate by Month 
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Notes: The distribution in Panel A shows the total number of interactions that students in the treatment group had 
with their advisor. The line plot in Panel B shows the share of treatment students who interacted with their advisor 
by month. We exclude students who have not yet signed up by that month from the interaction rate. 


Figure 3: Geographic Distribution of Experimental Sample 


Notes: The map shows the share of students in the experimental sample by state. 
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Figure 4: Distribution of Graduation Rates for College Decisions by Treatment vs Control Group 
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Notes: The figure plots the distribution of 6-year college graduation rates at the schools to which students in the 
control and CollegePoint treatment groups enrolled following high school. We restricted the sample to students who 
enrolled the semester following high school. 
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Appendix Table 1 


Intervention Dosage: Advising Usage by Cohort 


Cohort 1 

Mean N 

() (2) 
% Ever Interacted 77.7 6672 
Avg. Number of Interactions 5.8 6672 
% Discussed College List 40.9 6672 
% Discussed College Applications 57.6 6672 
% Discussed Applying for Financial Aid 60.1 6672 
% Discussed College Decision 39.0 6672 
% Discussed College Transition 23.5 6672 


Notes: The table shows usage of CollegePoint advising by cohort for students in the treatment group based on 


Cohort 2 

Mean N 
(3) (4) 
70.7 10198 
5.1 10198 
38.7 10198 
48.9 10198 
54.4 10198 
36.5 10198 
18.8 10198 


Cohort 3 

Mean N 
(5) (6) 
72.3 4019 
6.3 4019 
48.7 4019 
56.1 4019 
55.6 4019 
38.4 4019 
26.8 4019 
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Percentile College Entrance Exam Score 


% Family Income < $40,000 


% Family Income $40,000- $80,000 


% Family Income > $80,000 

% Asian 

% Black 

% Hispanic 

% White 

% Other Race 

% Parents without a BA 

% Female 

% Urban High School 

% Suburban High School 

% Rural High School 

% Public High School 

% Private High School 

% Charter High School 

% Magnet High School 

% Low-Supply CP School State 
% Mid-Supply CP School State 
% High-Supply CP School State 
School Size 

HS Student:Counselor Ratio 

% FRL 

HS Per Pupil Expenditure 


F-Test 


Cohort 1 
Pooled 
Mean N 
() (2) 
95.2 9249 
41.6 9249 
5.2. 9249 
1.2 9249 
26.9 8908 
7.7 8908 
19.3 8908 
40.7 8908 
5.3 8908 
53.8 8759 
54.1 9044 
38.4 9151 
48.6 9151 
13.0 9151 
774 9111 
6.7 9111 
4.0 9111 
12.1 9111 
11.0 9249 
21.7 9249 
67.3 9249 
1710 »=9111 
289.83 8501 
42.8 8333 
15765 8501 
F-Stat P-Val 
0.743 0.732 


Appendix Table 2 
Summary Statistics and Balance Tests for Students by Cohort 


T-C 

P-Val 
(3) 

0.162 
0.837 
0.908 
0.715 
0.238 
0.085* 
0.198 
0.391 
0.591 
0.966 
0.457 
0.736 
0.942 
0.704 
0.967 
0.593 
0.719 
0.700 
0.667 
0.965 
0.740 
0.645 
0.746 
0.568 
0.528 


N 
9249 


Cohort 2 
Pooled 
Mean N 
(4) (5) 
95.6 11824 
39.3. 11824 
59.0 11824 
1.7 11824 
24.3 11144 
6.3 11144 
19.1 11144 
44.5 11144 
5.8 11144 
51.5 10830 
55.6 11338 
36.2 11642 
48.2 11642 
15.6 11642 
77.0 11560 
7.9 11560 
4.4 11560 
11.0 11560 
15.1 11824 
24.7 11824 
60.3. 11824 
1644 11566 
292.79 10641 
41.6 10449 
15444 10642 
F-Stat P-Val 
0.685 0.791 


T-C 
P-Val 

(6) 
0.994 
0.120 
0.173 
0.475 
0.246 
0.907 
0.867 
0.241 
0.898 
0.413 
0.909 
0.276 
0.531 
0.561 
0.245 
0.809 

0.093* 
0.574 
0.387 
0.369 
0.145 

0.060* 

0.025** 

0.122 
0.239 


N 
11824 


Cohort 3 
Pooled 
Mean N 
(7) (8) 
42.7 4623 
57.3 4623 
26.5 3818 
9.9 3818 
27.4 3818 
30.1 3818 
6.1 3818 
50.2 3832 
57.3 3876 
39.1 4467 
47.4 4467 
13.5 4467 
76.0 4444 
6.0 4444 
4.7 4444 
13.5 4444 
11.1 4623 
21.5 4623 
67.4 4623 
1686 4446 
290.7. 4179 
45.6 4096 
15678 4179 
F-Stat P-Val 
2.783 0.025 


T-C 
P-Val 
(9) 


0.136 
0.136 


0.540 
0.530 
0.959 
0.526 
0.137 
0.507 
0.819 
0.692 
0.454 
0.352 
0.841 
0.170 
0.134 
0.610 


N 
3477 


Notes: This table presents baseline summary statistics for the students in the experimental sample by cohort. 
Due to changes with a data provider, we lack detailed student-level data for students from cohort 3. We present 
cohort means and counts for each cohort separately. In Column 3, 6, and 9 we show the p-value for a test of the 
difference between treatment and control group means, estimated by regressing the relevant baseline variable on 
the treatment group indicator and a randomization batch fixed effect. We report a joint balance test for each 
cohort by regressing treatment status on all baseline variables in the table excluding high school characteristics. 
The bottom of the table contains the resulting F-statistic and p-value. All regressions use robust standard errors. 


* p<0.1, ** p<0.05, *** p<0.01 
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Appendix Table 3 
Summary Statistics: Takers vs Non-Takers 


pe Take? Taker Diff SE P-Value 
Mean 
(1) (2) 3) (4) 
Panel A: Demographic Differences 
Percentile College Entrance Exam Score 95.6 0.4 (0.101) 0.000*** 
Predicted Income 60160 -20389 (634.299) 0.000*** 
% Asian 36.3 0.5 (0.977) 0.585 
% Black 5.9 1.6 (0.464) 0.001*** 
% Hispanic 23.6 6.9 (0.850) 0.000*** 
% White 295 -8.6 (0.943) 0.000*** 
% Other Race 43 0.1 (0.426) 0.872 
% Parents without a BA 70.2 3.0 (0.928) 0.001*** 
% Female 50.5 6.8 (1.019) 0.000*** 
% Urban High School 46.7 4.8 (1.017) 0.000*** 
% Suburban High School 42.0 -2.8 (1.009) 0.006*** 
% Rural High School 11.0 -1.1 (0.641) 0.077* 
% Low-Supply CP School State 7.8 -0.5 (0.586) 0.430 
% Mid-Supply CP School State 22.0 2.6 (0.841) 0.002*** 
% High-Supply CP School State 70.2 -1.2 (0.944) 0.188 
% HS Offers AP Courses 94.6 0.4 (0.549) 0.438 
HS Student:Counselor Ratio 301.0 -28.9 (5.239) 0.000*** 
% HS Receiving FRPL 46.9 1.9 (0.466) 0.000*** 
HS Per Pupil Expenditure 16605 1698 (150.009)  0.000*** 
Observations: 11950 
Panel B: Score Send Differences 
Total Applications 8.5 1.5 (0.256) 0.000*** 
% Applied to CollegePoint School 92.7 2.8 (1.052) 0.009*** 
% Applied to Barron's | 78.9 11.0 (1.646) 0.000*** 
% Applied to Barron's 2 74.2 2.4 (1.745) 0.166 
% Applied to Barron's 3 56.7 0.4 (1.972) 0.852 
% Applied to Barron's 4+ 61.3 -3.8 (1.934) 0.052* 
Observations: 9457 


Notes: This table compares the students who were recruited by CollegePoint but did not sign up ("non-takers") to 
students who signed up ("takers"). Panel A presents differences in baseline demographics and includes all non- 
takers and takers. Panel B presents differences in SAT score sends and includes non-takers and the control group 
takers. We estimate difference between takers and non-taker group means by regressing the relevant outcome 
variable on the take-up indicator. All regressions use robust standard errors. See Section III for more details. 

* p<0.1, ** p<0.05, *** p<0.01 
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Summary Statistics and Balance Tests for Supplemental RCTs 


Percentile College Entrance Exam Score 


% Family Income < $40,000 


% Family Income $40,000- $80,000 


% Family Income > $80,000 

% Asian 

% Black 

% Hispanic 

% White 

% Other Race 

% Parents without a BA 

% Female 

% Urban High School 

% Suburban High School 

% Rural High School 

% Public High School 

% Private High School 

% Charter High School 

% Magnet High School 

% Low-Supply CP School State 
% Mid-Supply CP School State 
% High-Supply CP School State 
School Size 

HS Student:Counselor Ratio 

% FRL 

HS Per Pupil Expenditure 


F-Test 


Appendix Table 4 


On-Track Sample 
Pooled T-C 
Mean N P-Value 
Ce — 20) 3) 
88.8 2965 0.359 
46.0 3394 0.648 
53.9 3394 0.623 
0.1 3394 0.566 
19.3 1915 0.147 
10.3 1915 0.763 
35.8 1915 0.005*** 
28.7 1915 0.332 
5.9 1915 0.277 
69.0 2158 0.745 
59.2 1936 0.223 
35.1 3323 0.200 
49.9 3323 0.272 
15.0 3323 0.870 
78.6 3305 0.615 
6.3 3305 0.367 
5.3 3305 0.634 
10.0 3305 0.775 
12.0 3393 0.330 
23.5 3393 0.806 
64.5 3393 0.645 
1606 3307 0.710 
299 3097 0.841 
48.3 3028 0.320 
14952 3097 0.309 
F-Stat P-Value N 
1.06 0.390 2965 


Early-Start Sample 


Pooled 
Mean N 
(4) (5) 
95.3 2798 
27.4 3101 
52.5 3101 
20.0 3101 
41.7 3039 
5.7 3039 
21.1 3039 
25.8 3039 
5.8 3039 
40.0 3011 
47.9 3081 
46.7 3026 
43.6 3026 
9.8 3026 
72.9 3010 
6.0 3010 
5.3 3010 
16.1 3010 
11.0 3101 
23.7 3101 
65.4 3101 
1914 3011 
330 2402 
46.3 2720 
14341 2400 
F-Stat P-Value 
0.558 0.888 


T-C 
P-Value 
(6) 
0.945 
0.994 
0.226 
0.127 
0.802 
0.996 
0.094* 
0.369 
0.479 
0.248 
0.221 
0.386 
0.108 
0.222 
0.382 
0.199 
0.215 
0.348 
0.099* 
0.763 
0.398 
0.983 
0.257 
0.399 
0.663 


N 
2798 


Notes: This table presents baseline summary statistics for the students in the supplemental experiments reported in 
section VI. We present sample means and counts for each experiment separately. In Column 3 and 6 we show the p- 
value for a test of the difference between treatment and control group means, estimated by regressing the relevant 
baseline variable on the treatment group indicator and a randomization batch fixed effect. We report a joint balance 
test for each experiment by regressing treatment status on all baseline variables in the table excluding high school 
characteristics. The bottom of the table contains the resulting F-statistic and p-value. All regressions use robust 


standard errors. 
* p<0.1, ** p<0.05, *** p<0.01 
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Campaign | Recruitment 
Campaign 2 Recruitment 
Campaign 3 Recruitment 


Percentile College Entrance Exam Score 


% Family Income < $40,000 


% Family Income $40,000- $80,000 


% Family Income > $80,000 

% Asian 

% Black 

% Hispanic 

% White 

% Other Race 

% Parents without a BA 

% female 

% Urban High School 

% Suburban High School 

% Rural High School 

% Public High School 

% Private High School 

% Charter High School 

% Magnet High School 

% Low-Supply CP School State 
% Mid-Supply CP School State 
% High-Supply CP School State 
School Size 

% FRL 

HS Student:Counselor Ratio 
HS Per Pupil Expenditure 


Notes: This table presents baseline summary statistics for CollegePoint students by advising organization. We 
present means and counts for each organization separately. 


Appendix Table 5 


Org | 
Mean N 
() (2) 
0.9 4370 
31.2 4370 
68.0 4370 
95.2 4370 
36.5 4370 
61.3 4370 
2.3 4370 
22.9 4155 
7.8 4155 
18.6 4155 
45.1 4155 
5.6 4155 
49.2 4006 
58.1 4229 
35.7 4307 
50.0 4307 
14.3 4307 
76.3 4287 
8.1 4287 
4.6 4287 
11.3 4287 
14 4370 
22.8 4370 
64 4370 
1608 4288 
42 3850 
289 3941 
15506 =3941 


Summary Statistics by Advising Organization 


Org 2 

Mean N 

G).— 
27.8 7890 
0.0 7890 
72.2 7890 
95.5 7890 
38.0 7890 
60.7 7890 
1.4 7890 
24.5 7454 
6.0 7454 
18.2 7454 
45.3 7454 
6.0 7454 
51.3 7241 
52.6 7579 
36.4 7769 
47.5 7769 
16.1 7769 
78.5 7726 
75 7726 
3.8 7726 
10.4 7726 
15 7890 
24.0 7890 
61 7890 
1648 7728 
41 7025 
293 7150 
15392 7150 


Org 3 
Mean N 
(5) (6) 
76.3 2139 
5.8 2139 
17.9 2139 
95.6 2139 
51.8 2139 
47.5 2139 
0.7 2139 
32.1 2062 
7.3 2062 
22.3 2062 
33.9 2062 
4.4 2062 
64.2 2085 
55.6 2093 
41.0 2121 
47.9 2121 
11.0 2121 
74.9 2107 
6.4 2107 
4.4 2107 
14.6 2107 
11 2139 
25.2 2139 
63 2139 
1820 2107 
45 1942 
291 1973 
15634 91973 


Org 4 

Mean N 
D3) 
3.3 1143 
84.1 1143 
12.6 1143 
95.6 1143 
48.7 1143 
50.8 1143 
0.4 1143 
29.9 1116 
73 1116 
24.1 1116 
34.2 1116 
4.4 1116 
52.9 1125 
55:5 1128 
39.0 1136 
49.6 1136 
11.4 1136 
77.6 1128 
7.0 1128 
3.5 1128 
12.4 1128 
7 1143 
19.4 1143 
74 1143 
1753 1128 
42 1024 
276 1049 
16646 1049 
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Student Experience with Advising by Organization 


Appendix Table 6 


Org 1 Org 2 Org 3 Org 4 
Mean N Mean N Mean N Mean 
C) (2) (3) (4) (5) (6) (7) 
Panel A: Advisor Relationship 
I wish my advisor knew me better 59 1425 46 2696 59 670 52 
I feel close to advisor 28 1423 46 2696 48 671 50 
When I'm stressed, I contact my advisor 16 1425 29 2696 30 669 31 
My advisor is easy to talk to 71 1418 85 2696 81 670 81 
My advisor really listens to me 70 1415 85 2689 82 666 82 
My advisor cares about me 70 1415 85 2687 80 663 79 
Panel B: Advisor Helpfulness 
Advisor helped complete a task 62 1431 78 2700 78 668 76 
Advisor informed of a task 69 1431 83 2705 80 669 77 
Advisor helped track deadlines 65 1428 79 2696 76 669 75 
Advisor helped with where to apply 52 1429 72 2699 73 669 68 
Advisor helped complete applications 50 1427 66 2690 67 665 67 
Advisor helped with college essays 46 1426 63 2695 67 667 70 
Advisor helped with applying for aid 64 1431 78 2700 70 669 66 
Advisor helped expand college list 45 1430 66 2703 66 666 62 
Advisor helped manage stress 55 1432 71 2698 68 668 68 
Advisor helped with family issues 34 1425 53 2690 50 665 47 
Advisor helped with understanding aid award letters 55 1427 71 2697 59 665 53 
Advisor helped with college decision 46 1430 64 2696 62 667 53 
Panel C: Advisor Influence 
Advisor influenced college list 32 1422 49 2683 50 669 46 
Advisor influenced college decision 22 1399 17 2627 44 659 17 


Notes: The table shows experience of CollegePoint advising for students in the treatment group by organization based student self-reports from a post- 
intervention survey. The sample includes students from cohorts 1 and 2 who responded to the survey. See section III for more details about the survey. 


Appendix Figure 1: CollegePoint Start Date by Cohort 
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Notes: The line plots show the cumulative share of students from a given cohort who signed up for CollegePoint by 
week. The vertical dashed lines represent the average start date for students from a given cohort. 
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Appendix Figure 2: Interactions with Advisor by Cohort 


A) Distribution of Interactions: Cohort 1 B) Distribution of Interactions: Cohort 2 
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D) Interaction Rate by Month 


60 


Share of Students who Interacted 
30 


a) 
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Notes: The distributions in Panel A-C show the total number of interactions that students in the treatment group had 
with their advisor by cohort. The line plots in Panel D shows the share of treatment students from a given cohort 
who interacted with their advisor by month. We exclude students who have not yet signed up by that month from the 
interaction rate. 


54 


Appendix Figure 3: Effect of Starting Early on Monthly Interaction Rate 
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Notes: The figure presents the share of students who engaged by month. The grey dashed line plots late-starters and 
the purple line plots sum of the late-starters and estimated early-late difference. The error bars indicate 95 percent 
confidence intervals. 
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